A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics.

During the diagnostic process, clinicians leverage multimodal information, such as the chief complaint, medical images and laboratory test results. Deep-learning models for aiding diagnosis have yet to meet this requirement of leveraging multimodal information. Here we report a transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model leverages embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and uses bidirectional blocks with intramodal and intermodal attention to learn holistic representations of radiographs, the unstructured chief complaint and clinical history, and structured clinical information such as laboratory test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary disease (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Unified multimodal transformer-based models may help streamline the triaging of patients and facilitate the clinical decision-making process.

A CT-Based Radiomics Model for Prediction of Prognosis in Patients with Novel Coronavirus Disease (COVID-19) Pneumonia: A Preliminary Study.

This study aimed to develop a computed tomography (CT)-based radiomics model to predict the outcome of COVID-19 pneumonia. In total of 44 patients with confirmed diagnosis of COVID-19 were retrospectively enrolled in this study. The radiomics model and subtracted radiomics model were developed to assess the prognosis of COVID-19 and compare differences between the aggravate and relief groups. Each radiomic signature consisted of 10 selected features and showed good performance in differentiating between the aggravate and relief groups. The sensitivity, specificity, and accuracy of the first model were 98.1%, 97.3%, and 97.6%, respectively (AUC = 0.99). The sensitivity, specificity, and accuracy of the second model were 100%, 97.3%, and 98.4%, respectively (AUC = 1.00). There was no significant difference between the models. The radiomics models revealed good performance for predicting the outcome of COVID-19 in the early stage. The CT-based radiomic signature can provide valuable information to identify potential severe COVID-19 patients and aid clinical decisions.

Self-paced Multi-view Learning for CT-based severity assessment of COVID-19.

Prior studies for the task of severity assessment of COVID-19 (SA-COVID) usually suffer from domain-specific cognitive deficits. They mainly focus on visual cues based on single cognitive functions but fail to reconcile the valuable information from other alternative views. Inspired by the cognitive process of radiologists, this paper shifts naturally from single-symptom measurements to a multi-view analysis, and proposes a novel Self-paced Multi-view Learning (SPML) framework for automated SA-COVID. Specifically, the proposed SPML framework first comprehensively aggregates multi-view contexts in lung infection with different measure paradigms, i.e., Global Feature Branch, Texture Feature Branch, and Volume Feature Branch. In this way, multiple-perspective clues are taken into account to reflect the most essential pathological manifestation on CT images. To alleviate small-sample learning problems, we also introduce an optimization with self-paced learning strategy to cognitively increase the characterization capabilities of training samples by learning from simple to complex. In contrast to traditional batch-wise learning, a pure self-paced way can further guarantee the efficiency and accuracy of SPML when dealing with small and biased samples. Furthermore, we construct a well-established SA-COVID dataset that contains 300 CT images with fine annotations. Extensive experiments on this dataset demonstrate that SPML consistently outperforms the state-of-the-art baselines. The SA-COVID dataset is publicly released at https://github.com/YishuLiu/SA-COVID.

The correlation study on chest CT features and kidney injury in severe COVID-19 pneumonia from a multicenter cohort study.

Background: Among confirmed severe COVID-19 patients, although the serum creatinine level is normal, they also have developed kidney injury. Early detection of kidney injury can guide doctors to choose drugs reasonably. Study found that COVID-19 have some special chest CT features. The study aimed to explore which chest CT features are more likely appear in severe COVID-19 and the relationship between related (special) chest CT features and kidney injury or clinical prognosis. Methods: In this retrospective study, 162 patients of severe COVID-19 from 13 medical centers in China were enrolled and divided into three groups according to the estimated glomerular filtration rate (eGFR) level: Group A (eGFR < 60 ml/min/1.73 m2), Group B (60 ml/min/1.73 m2 ≤ eGFR < 90 ml/min/1.73 m2), and Group C (eGFR ≥ 90 ml/min/1.73 m2). The demographics, clinical features, auxiliary examination, and clinical prognosis were collected and compared. The chest CT features and eGFR were assessed using univariate and multivariate Cox regression. The influence of chest CT features on eGFR and clinical prognosis were calculated using the Cox proportional hazards regression model. Results: Demographic and clinical features showed significant differences in age, hypertension, and fatigue among the Group A, Group B, and Group C (all P < 0.05). Auxiliary examination results revealed that leukocyte count, platelet count, C-reactive protein, aspartate aminotransferase, creatine kinase, respiratory rate ≥ 30 breaths/min, and CT images rapid progression (>50%) within 24-48 h among the three groups were significantly different (all P < 0.05). Compared to Group C (all P < 0.017), Groups A and B were more likely to show crazy-paving pattern. Logistic regression analysis indicated that eGFR was an independent risk factor of the appearance of crazy-paving pattern. The eGFR and crazy-paving pattern have a mutually reinforcing relationship, and eGFR (HR = 0.549, 95% CI = 0.331-0.909, P = 0.020) and crazy-paving pattern (HR = 2.996, 95% CI = 1.010-8.714, P = 0.048) were independent risk factors of mortality. The mortality of severe COVID-19 with the appearance of crazy-paving pattern on chest CT was significantly higher than that of the patients without its appearance (all P < 0.05). Conclusions: The crazy-paving pattern is more likely to appear in the chest CT of patients with severe COVID-19. In severe COVID-19, the appearance of the crazy-paving pattern on chest CT indicates the occurrence of kidney injury and proneness to death. The crazy-paving pattern can be used by doctors as an early warning indicator and a guidance of reasonable drug selection.

The correlation study on chest CT features and kidney injury in severe COVID-19 pneumonia from a multicenter cohort study

Background Among confirmed severe COVID-19 patients, although the serum creatinine level is normal, they also have developed kidney injury. Early detection of kidney injury can guide doctors to choose drugs reasonably. Study found that COVID-19 have some special chest CT features. The study aimed to explore which chest CT features are more likely appear in severe COVID-19 and the relationship between related (special) chest CT features and kidney injury or clinical prognosis. Methods In this retrospective study, 162 patients of severe COVID-19 from 13 medical centers in China were enrolled and divided into three groups according to the estimated glomerular filtration rate (eGFR) level: Group A (eGFR < 60 ml/min/1.73 m2), Group B (60 ml/min/1.73 m2 ≤ eGFR < 90 ml/min/1.73 m2), and Group C (eGFR ≥ 90 ml/min/1.73 m2). The demographics, clinical features, auxiliary examination, and clinical prognosis were collected and compared. The chest CT features and eGFR were assessed using univariate and multivariate Cox regression. The influence of chest CT features on eGFR and clinical prognosis were calculated using the Cox proportional hazards regression model. Results Demographic and clinical features showed significant differences in age, hypertension, and fatigue among the Group A, Group B, and Group C (all P < 0.05). Auxiliary examination results revealed that leukocyte count, platelet count, C-reactive protein, aspartate aminotransferase, creatine kinase, respiratory rate ≥ 30 breaths/min, and CT images rapid progression (>50%) within 24–48 h among the three groups were significantly different (all P < 0.05). Compared to Group C (all P < 0.017), Groups A and B were more likely to show crazy-paving pattern. Logistic regression analysis indicated that eGFR was an independent risk factor of the appearance of crazy-paving pattern. The eGFR and crazy-paving pattern have a mutually reinforcing relationship, and eGFR (HR = 0.549, 95% CI = 0.331–0.909, P = 0.020) and crazy-paving pattern (HR = 2.996, 95% CI = 1.010–8.714, P = 0.048) were independent risk factors of mortality. The mortality of severe COVID-19 with the appearance of crazy-paving pattern on chest CT was significantly higher than that of the patients without its appearance (all P < 0.05). Conclusions The crazy-paving pattern is more likely to appear in the chest CT of patients with severe COVID-19. In severe COVID-19, the appearance of the crazy-paving pattern on chest CT indicates the occurrence of kidney injury and proneness to death. The crazy-paving pattern can be used by doctors as an early warning indicator and a guidance of reasonable drug selection.

AI-based analysis of CT images for rapid triage of COVID-19 patients.

The COVID-19 pandemic overwhelms the medical resources in the stressed intensive care unit (ICU) capacity and the shortage of mechanical ventilation (MV). We performed CT-based analysis combined with electronic health records and clinical laboratory results on Cohort 1 (n = 1662 from 17 hospitals) with prognostic estimation for the rapid stratification of PCR confirmed COVID-19 patients. These models, validated on Cohort 2 (n = 700) and Cohort 3 (n = 662) constructed from nine external hospitals, achieved satisfying performance for predicting ICU, MV, and death of COVID-19 patients (AUROC 0.916, 0.919, and 0.853), even on events happened two days later after admission (AUROC 0.919, 0.943, and 0.856). Both clinical and image features showed complementary roles in prediction and provided accurate estimates to the time of progression (p < 0.001). Our findings are valuable for optimizing the use of medical resources in the COVID-19 pandemic. The models are available here: https://github.com/terryli710/COVID_19_Rapid_Triage_Risk_Predictor .

A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images.

Common lung diseases are first diagnosed using chest X-rays. Here, we show that a fully automated deep-learning pipeline for the standardization of chest X-ray images, for the visualization of lesions and for disease diagnosis can identify viral pneumonia caused by coronavirus disease 2019 (COVID-19) and assess its severity, and can also discriminate between viral pneumonia caused by COVID-19 and other types of pneumonia. The deep-learning system was developed using a heterogeneous multicentre dataset of 145,202 images, and tested retrospectively and prospectively with thousands of additional images across four patient cohorts and multiple countries. The system generalized across settings, discriminating between viral pneumonia, other types of pneumonia and the absence of disease with areas under the receiver operating characteristic curve (AUCs) of 0.94-0.98; between severe and non-severe COVID-19 with an AUC of 0.87; and between COVID-19 pneumonia and other viral or non-viral pneumonia with AUCs of 0.87-0.97. In an independent set of 440 chest X-rays, the system performed comparably to senior radiologists and improved the performance of junior radiologists. Automated deep-learning systems for the assessment of pneumonia could facilitate early intervention and provide support for clinical decision-making.

A New Technique for Diagnosis of Dental Caries on the Children's First Permanent Molar

The current researches have been shown high prevalence and incidence of children's teeth caries, especially for the first permanent molar, which might do a lot of harm to their general health. Fortunately, early detection and protection can reduce the difficulty of treatment and protect children's oral health. However, traditional diagnostic methods such as dentist's visual inspection and radiographic imaging diagnosis are non-automatic and time-consuming. Given the COVID-19 epidemic, these methods should not be taken into consideration, since they fail to practice social distancing and further increase the risk of infection. To address these issues, in this paper we propose a novel caries detection and assessment (UCDA) framework to achieve a new technique for fully-automated diagnosis of dental caries on the children's first permanent molar. Inspired by an efficient in-network feature pyramid and anchor boxes, the proposed UCDA framework mainly contains a backbone network that is initialized with ResNet-FPN, and two parallel task-specific subnetworks for region regression and region classification. Due to the lack of the image database, we also present a novel children's oral image database, namely "Child-OID", which comprises 1, 368 primary school children's oral images with standard diagnostic annotations and labels, to evaluate the effectiveness of our UCDA method. Experiments on the Child-OID database demonstrate that commonly occurring caries on the first permanent molar can be more accurately detected via the proposed UCDA framework. Database and code are available at https://github.com/GipinLinn/UCDA-and-Child-OID.git.